Rates of contraction of posterior distributions based on Gaussian process priors

We derive rates of contraction of posterior distributions on nonparametric or semiparametric models based on Gaussian processes. The rate of contraction is shown to depend on the position of the true parameter relative to the reproducing kernel Hilbert space of the Gaussian process and the small ball probabilities of the Gaussian process. We determine these quantities for a range of examples of Gaussian priors and in several statistical settings. For instance, we consider the rate of contraction of the posterior distribution based on sampling from a smooth density model when the prior models the log density as a (fractionally integrated) Brownian motion. We also consider regression with Gaussian errors and smooth classification under a logistic or probit link function combined with various priors.Comment: Published in at http://dx.doi.org/10.1214/009053607000000613 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org

Adaptive Bayesian estimation using a Gaussian random field with inverse Gamma bandwidth

We consider nonparametric Bayesian estimation inference using a rescaled smooth Gaussian field as a prior for a multidimensional function. The rescaling is achieved using a Gamma variable and the procedure can be viewed as choosing an inverse Gamma bandwidth. The procedure is studied from a frequentist perspective in three statistical settings involving replicated observations (density estimation, regression and classification). We prove that the resulting posterior distribution shrinks to the distribution that generates the data at a speed which is minimax-optimal up to a logarithmic factor, whatever the regularity level of the data-generating distribution. Thus the hierachical Bayesian procedure, with a fixed prior, is shown to be fully adaptive.Comment: Published in at http://dx.doi.org/10.1214/08-AOS678 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org

Empirical processes indexed by estimated functions

We consider the convergence of empirical processes indexed by functions that depend on an estimated parameter $\eta$ and give several alternative conditions under which the ``estimated parameter'' $\eta_n$ can be replaced by its natural limit $\eta_0$ uniformly in some other indexing set $\Theta$. In particular we reconsider some examples treated by Ghoudi and Remillard [Asymptotic Methods in Probability and Statistics (1998) 171--197, Fields Inst. Commun. 44 (2004) 381--406]. We recast their examples in terms of empirical process theory, and provide an alternative general view which should be of wide applicability.Comment: Published at http://dx.doi.org/10.1214/074921707000000382 in the IMS Lecture Notes Monograph Series (http://www.imstat.org/publications/lecnotes.htm) by the Institute of Mathematical Statistics (http://www.imstat.org

Rejoinder to discussions of "Frequentist coverage of adaptive nonparametric Bayesian credible sets"

Rejoinder of "Frequentist coverage of adaptive nonparametric Bayesian credible sets" by Szab\'o, van der Vaart and van Zanten [arXiv:1310.4489v5].Comment: Published at http://dx.doi.org/10.1214/15-AOS1270REJ in the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org

Frequentist coverage of adaptive nonparametric Bayesian credible sets

We investigate the frequentist coverage of Bayesian credible sets in a nonparametric setting. We consider a scale of priors of varying regularity and choose the regularity by an empirical Bayes method. Next we consider a central set of prescribed posterior probability in the posterior distribution of the chosen regularity. We show that such an adaptive Bayes credible set gives correct uncertainty quantification of "polished tail" parameters, in the sense of high probability of coverage of such parameters. On the negative side, we show by theory and example that adaptation of the prior necessarily leads to gross and haphazard uncertainty quantification for some true parameters that are still within the hyperrectangle regularity scale.Comment: Published at http://dx.doi.org/10.1214/14-AOS1270 in the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org

The Horseshoe Estimator: Posterior Concentration around Nearly Black Vectors

We consider the horseshoe estimator due to Carvalho, Polson and Scott (2010) for the multivariate normal mean model in the situation that the mean vector is sparse in the nearly black sense. We assume the frequentist framework where the data is generated according to a fixed mean vector. We show that if the number of nonzero parameters of the mean vector is known, the horseshoe estimator attains the minimax $\ell_2$ risk, possibly up to a multiplicative constant. We provide conditions under which the horseshoe estimator combined with an empirical Bayes estimate of the number of nonzero means still yields the minimax risk. We furthermore prove an upper bound on the rate of contraction of the posterior distribution around the horseshoe estimator, and a lower bound on the posterior variance. These bounds indicate that the posterior distribution of the horseshoe prior may be more informative than that of other one-component priors, including the Lasso.Comment: This version differs from the final published version in pagination and typographical detail; Available at http://projecteuclid.org/euclid.ejs/141813426

Bayesian inverse problems with Gaussian priors

The posterior distribution in a nonparametric inverse problem is shown to contract to the true parameter at a rate that depends on the smoothness of the parameter, and the smoothness and scale of the prior. Correct combinations of these characteristics lead to the minimax rate. The frequentist coverage of credible sets is shown to depend on the combination of prior and true parameter, with smoother priors leading to zero coverage and rougher priors to conservative coverage. In the latter case credible sets are of the correct order of magnitude. The results are numerically illustrated by the problem of recovering a function from observation of a noisy version of its primitive.Comment: Published in at http://dx.doi.org/10.1214/11-AOS920 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org

Exploration of the Neuronal Subtype Specificity of an Ethanol Responsive Gene: Glycogen Synthase Kinase 3 Beta (Gsk3b)

Exploration of the Neuronal Subtype Specificity of an Ethanol Responsive Gene: Glycogen Synthase Kinase 3 Beta (Gsk3b) Dalton Huey, Depts. of Bioinformatics, Biology & Chemistry, A. van der Vaart, G. M. Harris, and M. F. Miles, with Dr. Sarah Golding, Dept. of Biology Previous work done in our laboratory revealed that Glycogen Synthase Kinase 3 Beta (Gsk3b) functions as a hub gene in a network of genes regulated by acute ethanol in the medial prefrontal cortex (mPFC) across a mouse genetic panel. Adult mice treated with acute ethanol showed increased phosphorylation of GSK3B on the Ser9 residue in prefrontal cortex. Subsequent viral-mediated overexpression of Gsk3bin mouse mPFC caused an increase in ethanol consumption and pharmacological inhibition of GSK3B decreased ethanol consumption. However, it is unknown what neuron subtypes are driving this change in behavior. Here, we provide evidence that deletion of Gsk3bin Camk2a+ glutamatergic neurons of the mPFC results in a decrease in ethanol consumption in both continuous and intermittent access drinking paradigms. Furthermore, we have recently designed and validated a plasmid for Cre-dependent overexpression of Gsk3b, along with a Cre-dependent reporter as a control. These plasmids are planned for use in conjunction with different Cre drivers for viral-mediated expression in any cell type. Dissection of the neural circuitry of this ethanol responsive pathway can lead to a better assessment of Gsk3bas a potential target for the treatment of alcohol use disorders. Work supported by grants R01A027581 and P50AA022537 to MFM.https://scholarscompass.vcu.edu/uresposters/1313/thumbnail.jp

Observed Information in Semiparametric Models

Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/90945/1/observed_information_semi-parametric_models.pdf6512512

Existence and consistency of maximum likelihood in upgraded mixture models

AbstractSuppose one observes a sample of size m from the mixture density ∫ p(x|z) dη(z) and a sample of size n from the distribution η. The kernel p(x|z) is known. We show existence of the maximum likelihood estimator for η, characterize its support, and prove consistency as m, n → ∞